The Samba team is once again participating in the Summer of Code this year. This year we have 4 students working on various projects related to Samba. This year I am mentoring Dhananjay Sathe, who is improving the GTK+ frontends for Samba. In particular, he is making it possible to manage shares and users of a remote Samba or Windows machine. Dhananjay is also blogging about his progress.

It's been more than a month since the last status update on my Samba 4 work - much more than the two weeks I promised. During the holidays I finally managed to release the new alpha of Samba 4, as well as releases of some of our companion libraries (tdb, talloc, tevent and ldb). The release includes a significant amount of bug fixes and a lot of work towards a properly functioning Active Directory DC, too much to list here. This release I've mainly been involved in improving our Python bindings and our handling of internal and external libraries. We now use symbol versioning for our copy of Heimdal Kerberos as well as some of our other libraries. Hopefully this will fix some of the issues users of the evolution-mapi plugin have been seeing where they end up with both MIT Kerberos and Heimdal Kerberos loaded into the same process (with all the consequences of overlapping symbol names). Samba 4 now also has the ability to work with the system Heimdal rather than using the bundled copy. I have packaged alpha14 for Debian and Ubuntu (fixing most of the open bugs against the Samba 4 package in the BTS), but am currently waiting for the new release of ldb to pass through NEW before I can upload. The next release is scheduled for the first week of February. cp: Stream of Passion - Haunted

After Samba XP 2008 Andrew and I started keeping a wiki page with our bi-weekly goals and achievements for Samba 4. Because planning in a Free Software project is hard (time availability and priorities change over time, and other volunteers are equally unpredictable) we called this our "Fantasy page"; it listed things we wanted to work on next ("fantasies"), but reality being what it is we would usually actually end up working on something entirely different. We discussed our progress and new plans in - what I would now call - a bi-weekly standup call. There were several reasons for doing this. It gave us some sense of direction as well as a sense of accomplishment; a way to look back at the end of the year and realize how much we had actually achieved. Because Samba 4 is such a long term project (it is 7 years old at this point) it is easy to become disillusioned, to look back at a year of commits and to not see the gradual improvement, just the fact that there is no release yet. We managed to keep this up for two years, much longer than I had anticipated, and eventually started to slip last year. More recently Kai and Tridge have started to blog weekly about their efforts to make Samba 4.0 a reality and I'm going to join them by trying to blog regularly - every two weeks - about my contributions, even if there were none. In the next two weeks I plan to work on finally getting alpha 14 of Samba 4 out and on fixing the daily builds of Samba 4 and OpenChange for Ubuntu on Launchpad after we did a massive reorganization of the private libraries in Samba 4. cp: Zero 7 - Somersault

There's more good news on the OpenChange front. Julien has been working together with Wolfgang and Ludovic from Inverse recently to leverage the server-side support in OpenChange to provide native Exchange server support in SOGo. A couple of days ago we announced that there now is an initial version that allows the use of Outlook against a SOGo server through OpenChange.

<object data="http://www.youtube.com/v/oSZJ95YeXYE?fs=1&amp;hl=en_US&amp;hd=1" height="385" type="application/x-shockwave-flash" width="480"><param name="movie" value="http://www.youtube.com/v/oSZJ95YeXYE?fs=1&amp;hl=en_US&amp;hd=1"><param name="movie" value="http://www.youtube.com/v/oSZJ95YeXYE?fs=1&amp;hl=en_US&amp;hd=1"><param name="allowFullScreen" value="true"><param name="allowscriptaccess" value="always"></object> (alternatively you can find the screencast here) As far as I know, this is the first time it's possible to actually use Outlook clients with a non-Microsoft Exchange-compatible server without the need for plugins on the Outlook side. And it's all Free Software. Of course, this is just a preview, and not something we'd recommend everybody to run in production yet. But it's exciting to finally see this come together. We already have OpenChange packages in Debian and Ubuntu but I hope I can help get SOGo packaged for both distributions as well.

Daily builds As of a month ago there are Ubuntu archives with fresh packages of Samba 4 and Openchange, built on a daily basis day from the latest upstream revision. This means that it is now possible to run a version of Samba 4 that is less than 24 hours old, without having to know how to extract source code from the version control system that upstream is using, without having to know how to build and install an application from source, but perhaps most importantly: without having to go through the tedious process of manually updating the source code and rebuilding. OpenChange is tightly coupled to Samba 4, so installing a new version of OpenChange usually involves installing a new version of Samba 4 as well. To make matters more confusing, the two projects use different version control systems (Samba 4 is in Git, while OpenChange is in Subversion) and different build systems (Samba 4 uses waf, OpenChange uses autoconf and make). I have been involved in Samba 4 and OpenChange as an upstream developer and more recently also as a packager for both Debian and Ubuntu. As an upstream developer for both these projects it is important for me that users can easily run the development versions. It makes it possible for interested users to confirm the fixes for issues they have reported and to test new features. The more users run the development version, the more confident I can be as a developer that doing a release will not cause any unexpected surprises. As a packager it is useful to know when there are upstream changes that are going to break my package with the next release.

Recipes The daily builds work using so-called recipes which describe how to build a Debian source package from a set of Bazaar branches. For example, the Samba 4 recipe looks like this:

# bzr-builder format 0.2 deb-version 4.0.0~alpha14~bzr revno ~ppa revno:packaging + revno:debian 
lp:samba
merge debian lp:~samba-team/samba/unstable
merge packaging lp:~samba-team/samba/4.0-ppa-maverick

This dictates that a source package should be built by taking the upstream Samba branch and merging the Debian packaging and some recipe-specific tweaking. The last bit on the first line indicates the version string to be used when generating a changelog entry for the daily build. Every night Launchpad (through bzr-builder) merges these branches and attempts to build the resulting source package, e-mailing me in case of build problems. Generally I fix issues that come up by committing directly to upstream VCS or to the Debian packaging branch. There is no overhead in maintaining the daily build after I've set it up. For more information on creating source package recipes, see getting started.

Toolchain The entire toolchain that does the daily package builds for Ubuntu is Free Software, and I have contributed to various bits of that toolchain over the years. It's exciting to see everything come together.

Soyuz Launchpad consists of multiple pillars - one of those pillars is Soyuz, which I hack on as part of my day job at Canonical. Soyuz is responsible for the archive management and package building. Debian source packages (a combination of upstream source code and packaging metadata) get uploaded by users and then built for various architectures on our buildfarm and published to the Ubuntu archive or to users personal package archives.

Launchpad-code Another pillar of Launchpad is Launchpad-code, which is responsible for the hosting and management of version control branches. Launchpad users can either host their branches on Launchpad directly or mirror branches (either native Bazaar branches or branches in a foreign format such as Subversion, Git or Mercurial). The mirrorring of native and foreign branches happens using standard Bazaar API's. In the case of Samba and OpenChange we import the branches of the upstream projects (Samba is in Git, OpenChange is in Subversion) and the packaging for both projects is in Bazaar. Launchad-code calls out to Bazaar to do the actual mirrorring. Over the last few years I have done a lot of work to improve Bazaars support for foreign branches, in particular on supporting Subversion, Git and Mercurial. As the code mirrorring in Launchpad is one of the biggest users of bzr-svn and bzr-git it has helped find some of the more obscure bugs in those plugins over the last few years, to the point where there are only a handful of issues with Git and Subversion imports left.

bzr-git and dulwich bzr-git provides transparent access to Git repositories from within Bazaar and is built on top of Dulwich. Dulwich is a Python library that provides access to the Git file formats and protocols that is completely independent of Bazaar. James Westby originally started it and I adopted it for bzr-git and further extended it. There are now several other projects that use it as well, including hg-git, and rabbitvcs. Apart from James and me almost two dozen other people have contributed it so far.

bzr-svn and subvertpy bzr-svn provides transparant access to Subversion repositories in Bazaar. When I grew frustrated with the existing Subversion Python bindings for various reasons, I decided to create independent Python bindings for Subversion from scratch. These bindings have since been split out into a separate project - subvertpy - and other projects have since also started using them, e.g. hgsubversion and basie.

Using the daily builds To use the Samba 4 and OpenChange daily builds (Ubuntu Maverick only for now), run:

$ apt-add-repository ppa:samba-team/ppa
$ apt-add-repository ppa:openchange/daily-builds

cp: Karnivool - Themata

Seeing this makes me very happy. It's taken us a couple of years to get to this point but we've finally made it, mostly thanks to the dedication and persistence of Julien and Brad.

Robert Collins did a very nice writeup not so long ago of creating a Debian package with bzr-builddeb, but he starts from the assumption that upstream also uses bzr. As we all know, there are many projects that have their upstream sources in a VCS, but don't use bzr. Can we still get reasonable results from bzr-builddeb, including the ability to merge new upstream releases direct from the VCS, if, say, upstream is using git? Yes, we can! Of course, if you're happy with using git as the VCS for your Debian packaging already, there's probably no compelling reason for you to switch to bzr. But if you prefer to use bzr and have been frustrated at having to choose between being able to use the VCS client of your choice for your packages and being able to use your VCS client to access upstream revision history, read on. Thanks to the fine work of Jelmer Vernooij and friends, bzr-git has been usable for a while if you just want to track the default git branch. But what if you care about tracking multiple upstream branches? Enter bzr git-import, which we'll use to set up a bzr repo from scratch for our cifs-utils package, with a full import of all the branches of upstream's git tree. First we do some work to set up a shared bzr repository. As long as we're going to the effort of mirroring all the git branches, we probably want to publish them for other people to use, so we make sure our bzr repository is reasonably configured for sharing data between branches:

$ bzr init-repo --no-trees cifs-utils.server
Shared repository (format: 2a)
Location:
  shared repository: cifs-utils.server
$ cd cifs-utils.server

Then we use bzr git-import, provided by the bzr-git plugin, to do a full import of the upstream branches into a cleverly named subdirectory:

$ bzr git-import git://git.samba.org/cifs-utils.git upstream
[/                   ] Counting objects: 181, done.

Now we create our own local 'trunk' for the package, by branching from the upstream tag matching the release version we're going to package.

$ bzr branch -r tag:cifs-utils-4.0rc1 upstream/HEAD trunk
Branched 42 revision(s).

Do a little more prep work for future branches...

$ mkdir branches

And that's it. Now we have a bzr repo locally that we can push out to our hosting server of choice (N.B.: we could do this all directly on the server, but alioth doesn't currently have bzr-git installed):

$ rsync -az . alioth.debian.org:/bzr/pkg-samba/cifs-utils/
$

Now that we have a bzr repository, it's time to get ourselves a working directory and do some packaging. We're probably going to work with multiple branches locally as well, so we create another shared repository for local use, and get a copy of the trunk branch we created before.

$ bzr init-repo cifs-utils
Shared repository with trees (format: 2a)
Location:
  shared repository: cifs-utils
$ cd cifs-utils
$ bzr co bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/trunk

Now we grab the upstream tarball, and whip up some quick packaging with (what else?) debhelper 7:

$ wget ftp://ftp.samba.org/pub/samba/cifs-utils/cifs-utils-4.0rc1.tar.bz2
$ ln -s cifs-utils-4.0rc1.tar.bz2 cifs-utils_4.0~rc1.orig.tar.bz2
$ cd trunk
$ mkdir -p debian/source
$ echo '3.0 (quilt)' > debian/source/format
$ echo 7 > debian/compat
$ cp /usr/share/doc/debhelper/examples/rules.tiny debian/rules
$ dch --package cifs-utils --versio 4.0~rc1-1 --create 'Initial package'

Create debian/control by hand, generate an initial source package, and import it into bzr with bzr-builddeb.

$ debuild -uc -us -S -i
$ rm -r debian
$ bzr import-dsc ../*.dsc
$

Since we want to continue tracking upstream development, we need to periodically sync our bzr import of the git tree.

$ bzr git-import git://git.samba.org/cifs-utils.git bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/upstream
$

And when we find a new upstream version in that import that we want to package, bzr-builddeb makes this a snap, too.

$ cd cifs-utils
$ wget ftp://ftp.samba.org/pub/samba/cifs-utils/cifs-utils-$version.tar.bz2
$ cd trunk
$ bzr merge-upstream --v3 --version $version_with_epoch \
    ../cifs-utils-$version.tar.bz2 -r tag:cifs-utils-$version \
    bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/upstream/HEAD
No distribution specified, and no changelog, assuming 'debian'
Committing to: /tmp/tmpbzOVMs/upstream/
Committed revision 2.
All changes applied successfully.
The new upstream version has been imported.
You should now review the changes and then commit.
$ bzr diff
<review changes>
$ bzr commit -m "merge upstream $version"
Committing to: bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/trunk
modified debian/changelog
Committed revision 50.
$

And that's it! It's exciting to see signs of real interoperability between DVCSes at last. Thanks to everyone who's helped make it possible!

From left to right: Obey Arthur Liu, Olly Betts, Stefano Zacchiroli, Dirk Eddelbuettel, Sylvestre Ledru, Jelmer Vernooij.

Dear Planet,

We arrived at the Google Summer of Code 2009 Mentor Summit and are having a blast here. The weather is awesome, the candies are plenty and the conference rooms are comfy at the Googleplex. We will write to you again soon.

Cheers

The Debian people Arthur, Olly, Zack, Dirk, Sylvestre, Jelmer

After over 7 years of working on it off and on, I'm looking for somebody to help maintain (and eventually take over) CtrlProxy. I started working on CtrlProxy somewhere in 2002, only a short while after Wilmer started hacking on BitlBee. If I remember correctly I started working on it because I didn't want to run a separate dircproxy (the only real competitor at the time) instance (with configuration) for each IRC network that I connected to. It was also just a good excuse to play with the IRC protocol a bit. Over the years, CtrlProxy has served as a playground for me to try out new and interesting things. It's been rewritten or severely refactored several times in its early history, the latest time being the 3.0 release (from 2005). I've tried different build systems, I've tried different implementation languages, I've tried different configuration file formats, I've tried different support libraries, I've tried different version control systems, I've tried different documentation formats. So while it's definitely been a very educational project for me personally, I haven't really had the time or the interest to dedicate to the project that it deserved during the last few years. This was mostly because there were other more interesting FOSS projects I spent my spare cycles on. These days there are plenty of other good IRC proxies out there, including BIP, ZNC and ShroudBNC so I doubt CtrlProxy will be missed if it were to disappear. Despite that, if anybody is interested in taking over, please send me an email (jelmer@samba.org) or contact me on IRC (jelmer on the OFTC and Freenode networks). cp: Anathema - Shroud of False

I will be attending Debconf 9 in Spain from July 23-30. I will also be attending debcamp the previous week. I m hoping to build contacts and increase my involvement in the Debian community, and the previous debconf I attended was an interesting window into what was going on in Debian and Linux. I m still lining up things to do at Debcamp. Jelmer Vernooij will be there; he s interested in working with me on Samba 4 support for MIT Kerberos in Debian. I m interested in working with him on making the user experience good for people who use both Samba 4 and other Kerberos applications. As I wrote at the bottom of this post, I believe it is critical that the open source community not just follow what Microsoft is doing in the Enterprise space. I also think it is important that we maintain avenues for our own innovation. To that end, I want to look at what we can do to use enterprise infrastructure independent of AD-look-alike projects like Samba as well. So, I ll be looking at making what I can do to help this in Debian. Areas of interest include:

1. Easy set up of Kerberos to use an LDAP database
2. Easy configuration of libpam-krb5 and libpam-ldap together using Kerberos for authentication and LDAP for authorization but not authentication.
3. Support for FAST integrated into Debian systems so we can gain better protection against weak passwords. As I promised, more about this in its own post.
4. Better support for PKI/smart cards for network authentication.

These are all projects I think I could make headway on myself. However the value of debcamp is the other people there. I ve never been to a debcamp before and so I don t know what it will be like. I do know that I will give higher priority to projects that will benefit from close cooperation over a week. So, if you re there and want to try to recruit me to your project, feel free. I m interested in enterprise infrastructure, VOIP, IPv6, network security and making complex infrastructure easy to use.

Some (not so) short news from the samba packaging team... Recent package uploads:

• Last "stable" upstream branch: 3.3.4. 3.3.4-2 is in unstable, quietly ageing before entering testing. 3.3.4-1 is in testing. In short, we have the latest upstream in squeeze
• Next upstream version: 3.4.0~pre1 is in experimental. That includes many packaging changes as upstream is now providing samba3 and samba4 altogether, which lead to source tree reorganization. Good news: there are no more "FHS" patches and upstream included needed changes for the file layout to finally be able to cope with the FHS entirely.
• We maintain many backports:
  • 3.3.4 was just uploaded to lenny-backports on backports.org. Maybe soon to come a 3.3.4 upload for etch-backports
  • unofficial backports on http://pkg-samba.alioth.debian.org: here we will maintain "the latest upstream version of the same upstream branch than what we have in Debian stable suites". That seems complicated to say and an example says it better: lenny has 3.2.5, so this repository has upstream 3.2.11 with the same packaging than 3.2.5. We made an exception for etch: instead of backporting the latest 3.0 upstream (etch has 3.0.24, upstream has 3.0.34), we backported the latest 3.2: we consider that nobody is now seriously considering to run 3.0, except those people who are very happy with the official packages in etch.
  • All these backports will be useful when tracking down bugs and ask our users if they reproduce them with the latest upstream versions, as samba setups are often very tricky to reproduce in our own environments.

Other things are coming, such as the work on having samba3 and samba4 packages coexist. Jelmer Vernooij prepared everything after our discussions at the SambaXP conference and it should be uploaded "soon". A few patches we have should be integrated upstream as well, therefore making the Debian diff smaller and smaller. Also, Luk Claes joined the maintenance team and helped setting up the unofficial repository for backports. The bug count is going up again these days as I don't spend much time triaging the bugs and some are very tricky to reproduce. As usual, help would be welcomed. I'm fairly sure that several of these are user errors but I often lack time to prove this enough for closing the bug. We should go to to 50-60 bugs or so.

... is to have upstream do it..:-) Apparently, yesterday, Jelmer Vernooij, member of the Samba Team and maintainer of samba4 and related packages (openchange, etc.) started some triaging of bugs of the samba package. After several years of efforts by Steve Langasek, No l K the and /me, the bug log of samba is not in a so bad shape, but, still, tseeing one of our upstream digging in our dirt is very good news... Thanks again, Jelmer. You really deserve to reach the end of your NM process, now.

The (still alpha) 4.0.0 release of Samba is now in experimental. Both Samba 3 and Samba 4 packages are likely to coexist in the archive as Samba 3 is still for a while the production code for File and Print services while Samba 4 is the development branch of the Samba team to implement Active Directory domain control (among other things). Samba4 packages also bring a big bunch of libraries, most of which being used to related development such as Openchange. Expect some neat new packages in experimental, then unstable, pretty soon now. Please do NOT replace production servers running samba 3.0.* or samba 3.2.* by samba4....not yet..:-) The samba4 packages are maintained by Jelmer Vernooij, one of the two main developers for Samba 4 in the Samba Team (the other being Andrew Bartlett). Jelmer is someone we should stuck out from the NM queue as soon as possible..:-) Anyway, this is again another success of that small team we built around Samba and samba-related packaging. Thanks to all involved people (Steve Langasek, No l K the, Jelmer Vernooij, Peter Eisentraut, etc.).

There has been some hype around samba this week. On May 21st, the Samba Team released Samba 3.0.29, an update from their stable branch. The Debian package maintainers quickly built and uploaded a package for unstable/lenny the very same day. Then, on 27th, we got notified of CVE 2008-1005, which affects both etch, lenny and unstable (not the 3.2 versions we have in experimental). I contacted the security team and, after some brief discussions, I uploaded with their blessing, a fixed package to stable-security (3.0.24-6etch10). Then, I immediately uploaded samba 3.0.30 to unstable (1:3.0.30-1), that version being the Samba Team response to the security issue. All this lead to DSA 1590. Up to now, everything perfect. In the meantime, Karolin, the Samba Team's release manager announced the Releace Candidate 1 of Samba 3.2.0. So, yesterday (Saturday), I started building it....which succeeded. Then I started to screw up..:) Instead of uploading that version to experimental, as planned, I uploaded 1:3.2.0~rc1-1 to unstable. Yes, AGAIN (I already did that in April). There's no excuse for that, except distraction and failure to use our tools properly ("dch -r"). Thankfully, I noticed that nearly immediately, but not immediately enough. So, I had to re-upload 3.0.30 to unstable (which is urgent: that's a security fix) and increase the epoch (we already had one as this is the second identical screwage): 2:3.0.30-2. Sorry, autobuilders... Yesterday evening, I uploaded 3.2.0-rc1 again to experimental (2:3.2.0~rc1-2). Then, finally, I went on the last bit of remaining work: spend some time on Jelmer Vernooij's samba4 packages. Jelmer is one of the two lead developers of Samba 4, a huge masterpiece still under heavy delveopment. To give it more exposure, it is planned to upload it a few times in experimental, then move it to unstable, block it so that it does not enter testing, and make some noise about this. I finally uploaded samba4 this morning (triple checking that I was uploading to experimental!) and samba4 4.0.0~alpha4~20080522-1 is now waiting for NEW processing (which might take time: this is a big piece of new stuff). I'm sad for the screwage because, apart from it, that was nearly a perfect process. But, anyway, I'm still proud of the result: 4 successful uploads for samba in a little more than a week (which included 4 day away). It could even have been 5 as the samba version in sarge is affected by the security issue, but we don't support sarge anymore.

I spent the entire week at the SambaXP conference in G ttingen, Germany. The conference is the annual conference of the community of developers, contributors and users of Samba. I general call it "my annual german pilgrimage" as I attended all seven editions of the conference, the only FLOSS conference I attend on my work time, as my daily work involves quite a lot of uses of samba. This year featured a workshop or training session held by John Terpstra, longstanding FLOSS evangelist and member of the Samba Team for over 10 years, and Karolin Seeger, the brand new release manager of Samba. This has indeed been an incredible opportunity to have discussions with them about the packaging work for Samba. Actually, the work we did in the last 3 years to bring Samba Debian packages in a very good shape (when I don't screw up) is much appreciated. I think these days have been another opportunity to keep that link very closed. I have been impressed by the promising work of Karolin with respect to the preparation of the release and the very serious way she has to do that work....as well as her very friendly, while still discreet approach to technical discussion. Some work was done on Samba package bugs, though less than usually (the remaining ones are harder and harder to tackle!). I mostly work on Debian packages as well as .deb packages prepared by Sernet (the services company that organises the conference and employer of some Samba Team members), as these could some day become the packages provided by the Samba Team. Talks at the conference were pretty interesting, to keep connected with Samba 3.2 and 4.0 development. News from Andrew Tridgell about exchanges with Microsoft (yes, The Evil) and access to MS documentation, are very promising. The collaboration between Samba developers and Microsoft enginneers is now working well again, at the engineers level (as Tridge says: "lawyers are away, now, we can talk at the engineers level and restore the link that existed in the early 90's"). I also could measure the progress of the Openchange project whose ultimate aim is to provide a complete Microsoft Exchange replacement solution. They currently have working MAPI libraries and an Evolution plugin for Exchange is under development, while the bricks to build a server are patiently being put together. Good discussions with Julien Kerihuel, the lead (and French) developer and manager of the project. Jelmer Vernooij and I also settled the final plans to get Samba4 packages in Debian. I proposed to make a first upload to experimental, but quite soon to upload to unstable, the point being a largest as possible exposure of that code, so that upstream developers (Jelmer himself, Andrew Bartlett and a few others...) get as much feedback as possible. That upload will not be targeted for lenny (we'll block it from entering testing). For that reason and also because Samba 3 and 4 will certainly coexist for years, the source package will be named "samba4". Expect more news quite soon. And final conclusion of that week: I've also been delighted to be able to visit Alex and Meike in Hildesheim, as well as Andreas in Wernigerode. It's always a great pleasure to see longstanding Debian friends. At the beginning of the week, I planned to see *two DD and finally, at the end of the week, I can tell I've seen *three* Debian developers, finally!